mHealth hyperspectral learning for instantaneous spatiospectral imaging of hemodynamics

Hyperspectral imaging acquires data in both the spatial and frequency domains to offer abundant physical or biological information. However, conventional hyperspectral imaging has intrinsic limitations of bulky instruments, slow data acquisition rate, and spatiospectral tradeoff. Here we introduce hyperspectral learning for snapshot hyperspectral imaging in which sampled hyperspectral data in a small subarea are incorporated into a learning algorithm to recover the hypercube. Hyperspectral learning exploits the idea that a photograph is more than merely a picture and contains detailed spectral information. A small sampling of hyperspectral data enables spectrally informed learning to recover a hypercube from an RGB image. Hyperspectral learning is capable of recovering full spectroscopic resolution in the hypercube, comparable to high spectral resolutions of scientific spectrometers. Hyperspectral learning also enables ultrafast dynamic imaging, leveraging ultraslow video recording in an off-the-shelf smartphone, given that a video comprises a time series of multiple RGB images. To demonstrate its versatility, an experimental model of vascular development is used to extract hemodynamic parameters via statistical and deep-learning approaches. Subsequently, the hemodynamics of peripheral microcirculation is assessed at an ultrafast temporal resolution up to a millisecond, using a conventional smartphone camera. This spectrally informed learning method is analogous to compressed sensing; however, it further allows for reliable hypercube recovery and key feature extractions with a transparent learning algorithm. This learning-powered snapshot hyperspectral imaging method yields high spectral and temporal resolutions and eliminates the spatiospectral tradeoff, offering simple hardware requirements and potential applications of various machine-learning techniques.Comment: This paper will appear in PNAS Nexu

<b>BRIDGING COLOR TO SPECTRUM FOR BIOPHOTONICS</b>

Advancements in machine learning are narrowing the gap in visual capabilities between machines and healthcare professionals, resulting in a transformation of the way we understand and address health challenges. Despite these advances, underlying limitations persist in addressing real-world problems, particularly in the precise capture of biological and physiological information. This is primarily because traditional trichromatic cameras fall short of representing reflectance spectra due to their limited spectral information. To overcome these limitations, hyperspectral imaging has emerged as a powerful tool for biomedical applications. By collecting a wealth of information at different wavelengths, hyperspectral imaging provides a comprehensive view of electromagnetic spectra, allowing non-invasive clinical analysis for accurate diagnostics. Snapshot hyperspectral imaging, in particular, is a competitive alternative to traditional cameras as it can capture a hyperspectral image in a single shot without the need for scanning individual wavelengths. Here, we introduce a computational snapshot hyperspectral imaging method, achieved through the integration of a machine learning approach with a streamlined optical system. We design an explainable machine learning algorithm by incorporating optical and biological knowledge into the algorithm. Therefore, the algorithm can reconstruct hyperspectral images with high spectralspatial resolution comparable to those of scientific spectrometers, despite the use of sparse information captured from the optical system. To demonstrate its versatility in biomedical applications, we extract hemodynamic parameters of peripheral microcirculation from embryonic model systems, tissue phantom samples, and human conjunctivas. Furthermore, we validate high accuracy of the results using conventional hyperspectral imaging and functional near-infrared spectroscopy. This learning-powered imaging method, characterized by high resolution and simplified hardware requirements, has the potential to offer solutions for various biomedical challenges by surpassing the constraints of conventional cameras and hyperspectral imaging.</p